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This study researches whether the ePortfolio is a suitable instrument for human capital management 
in the business environment. The implementation of ePortfolio systems in five different 
organizations is analyzed. It considers whether ePortfolio implementations were successful, and 
relevant critical success factors were identified. For the latter purpose, a theoretical framework for 
analysis was compiled from the literature. The results show that the ePortfolio proved to be a useful 
tool for HCM purposes in two cases. The ePortfolio enabled these organizations to enhance their 
talent management and performance appraisal practices. Three out of five cases failed, reaching a 
bare minimum of their company goals and objectives. To explain these findings, the implementation 
processes in each of the five cases was analyzed by means of a compiled theoretical framework of 
critical success factors. The empirical results led to a revision of the framework, identifying eleven 
critical success factors. These factors revolve around linking the ePortfolio with business objectives, 
carefully identifying information requirements and selecting a suitable system, actively managing 
the implementation by appropriate and dedicated staff throughout the organization, and ensuring that 
employees have ownership over their ePortfolio profiles. 


As a result of contemporary shifts in worker 
demographics and structural changes to the nature of 
work, there is a need for firms to prepare and utilize 
their workforce in an appropriate manner. Therefore, in 
recent years human resource management (HRM) 
scholars have paid increasing attention to the 
management of human capital (HCM) in organizations 
throughout the entire employee life cycle, with a 
growing emphasis on learning and development, 
performance and incentive management, employee 
retention, workforce planning, and worker 
redeployment (Schweyer, 2010). 

Despite often-heard claims that human capital is 
the most valuable resource of an organization, firms 
that proactively act on this notion by implementing 
HCM processes (e.g., strategic workforce planning) 
are few and far between (Huber, 2012). In 2013, 
CedarCrestone’s 16th Annual Human Resources 
(HR) Systems Survey revealed that only 14% of 
respondents reported adoption of such processes 
(Martin, 2013). This can be explained by the fact that 
the HR profession is still developing an acquaintance 
with evidence-based management and by a lack of 
research on how to utilize existing information 
systems accurately and effectively for this purpose 
(Huber, 2012). Furthermore, the management of 
human capital is an intricate and complex process 
(Schweyer, 2010) that depends on the gathering and 
analysis of reliable, qualitative data about an 
organization’s workforce. In the process of HCM, 
the gathering and interpretation of qualitative (e.g., 
competences-based information) and quantitative 
data (e.g., turnover rates) are vital (Pfeffer & Sutton, 
2006; Rousseau, 2006; Westphalen, 1999). Empirical 
research has shown, however, that many 
organizations struggle to gather qualitative data on 


employee performance, potential, and competences 
(Lukaszewski, Stone, & Stone-Romero, 2008). 

Barker (2003) proposed that ePortfolios are a 
feasible instrument in this process, due to their close 
linkage with the tracking and development of human 
capital, lifelong learning, and the assessment of prior 
learning by gathering qualitative data about individuals. 
ePortfolios possess unique properties that differentiate 
them from more commonly used human resource 
information systems. As a digital professional profile of 
an employee, an ePortfolio enables the capturing and 
comparison of qualitative data regarding the skill level 
and competences of employees, their ambitions, 
developmental potential, and career expectations (JISC, 
2009; Smith, 1996; Woodbury, Addams, & Neal, 
2009). Furthermore, in current times multiple career 
shifts are increasingly common. Therefore, there is a 
need for a so-called “career-passport,” which 
professionals carry with them as they move from one 
setting to another (Clark & Eynon, 2009). The 
ePortfolio can facilitate this. 

To this day, the ePortfolio has not been researched 
empirically in the organizational setting. The concept, 
which originated in the educational context, has 
however been argued to be suitable for professional 
purposes by many scholars in the ePortfolio domain 
(Balaban, Divjak, & Mu, 2011; Cambridge, 2010; 
Flanigan & Amirian, 2006; Greenberg, 2004; Jafari & 
Greenberg, 2003; JISC, 2009; Tosh & Werdmuller, 
2004). This study aims to substantiate these claims by 
evaluating empirically the implementation of ePortfolio 
systems in organizations in an innovative three-year 
program called “Let’s Connect,” which took place from 
early 2012 until late 2014. The program aimed at 
enhancing the mobility of workers, both within and 
across organizations, by introducing an ePortfolio. 
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The aim of this research is to investigate the 
feasibility of ePortfolio use in the workplace. The 
objectives include establishing whether the 
implementations were successful and identifying the 
critical success factors. In order to achieve the research 
objectives, the following research questions were 
formulated: 

1. To what extent were the ePortfolio 

implementations successful? 

2. What critical success factors are associated 
with the implementation of an ePortfolio 
system in organizations? 

Human Resource Information Systems 

A human resource information system is a 
“technology-based system used to acquire, store, 
manipulate, analyze, retrieve, and distribute pertinent 
information regarding an organization’s human 
resources” (Tannenbaum, 1990, p. 27). Examples of 
such systems include SAP and Oracle. A human 
resource information system comprises a database of 
performance-related information (Kavanagh & Thite, 
2009; Kovach & Cathcart, 1999). The information 
relates to such aspects as recruitment, training and 
development, performance evaluations, and turnover 
rates (DeSimone, Werner, & Harris, 2002). Utilizing 
this information, organizations can effectively manage, 
develop, and deploy their human capital (Bassett, 
Campbell, & Licciardi, 2003; Lawler & Mohrman, 
2004). Therefore, human resource information systems 
have been deemed critical contemporary HRM tools 
that enable organizations to transform data into critical 
business information (Marler & Floyd, 2009). 

ePortfolio 

ePortfolios are “personalized Web-based 
collections of work, responses to work, and reflections 
that are used to demonstrate key skills and 
accomplishments for a variety of contexts and time 
periods” (Lorenzo & Ittelson, 2005, p. 2). The 
information in an ePortfolio typically relates to work 
experience, ambitions, and acquired and developed 
competences and may include assessment results, 
research papers, certificates, reports on projects and 
teamwork, or internships (Flanigan & Amirian, 2006; 
Greenberg, 2004; Tosh & Werdmuller, 2004). 

The ePortfolio is a broad concept with a wide array 
of definitions and described purposes. This is the result 
of two conflicting paradigms that surround the 
ePortfolio concept: the constructivist and positivist 
approaches. The constructivist approach emphasizes the 
learner’s perspective and the importance of reflection 
and considers the ePortfolio as 


a learning environment in which the learner 
constructs meaning. This learning environment as 
given shape through the process of portfolio 
construction assumes that meaning varies across 
individuals, over time, and with purpose. The 
portfolio presents process, a record of the processes 
associated with learning itself; thus a summation of 
individual portfolios would be too complex for 
normative description. (Paulson & Paulson, 1994, 
p. 36) 

The positivist approach, on the other hand, considers 
the ePortfolio a tool to assess externally defined 
learning outcomes and “assumes that meaning is 
constant across users, contexts, and purposes” (Paulson 
& Paulson, 1994, p. 36). The latter approach was 
central to the ePortfolio implementations in the Let’s 
Connect program, due to the necessary evaluations of 
employee performance by certain organizational 
standards. 

ePortfolio Implementations 

While there is no existing framework for the 
evaluation of ePortfolio implementations in the 
workplace, there is a substantial amount of research 
towards ePortfolio implementations within educational 
institutes. The Learning Sciences Research Institute at 
Nottingham University developed the ePortfolio 
Maturity Model in 2007 to aid the monitoring of 
implementations (Hartnell-Young et ah, 2007). This 
model can be found in Appendix A. 

The ePortfolio Maturity Model (EMM) consists of 
factors that reflect the readiness of an educational 
organization to engage effectively in ePortfolio use. 
Such a model does not exist for ePortfolio use in the 
workplace. However, there has been substantial 
research towards the implementation of HR information 
systems in the workplace that can be used to 
complement this model. In the literature, this is better 
known as the critical success factors approach. One of 
the most cited definitions was introduced by Rockart 
(1979), who stated that critical success factors are “the 
limited number of areas in which results, if they are 
satisfactory, will ensure successful competitive 
performance for the organization” (p. 85). Today, 
information system experts increasingly use these 
factors to support the planning process (Esteves, 2004). 
The eight critical success factors identified in by 
Rockart and Delong (1988) have been reconfirmed by 
various other researchers (e.g., Bird, 1991; McBride, 
1997; Paller & Laska, 1990; Watson, Rainer, & Koh, 
1995). Poon and Wagner (2001) identified two 
additional factors from the literature, which resulted in 
the Executive Information System Success Factors 
(ESF) framework, as presented in Appendix B. 
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Method 

This research was conducted in accordance with 
the interpretivist paradigm and followed a two-stage 
qualitative approach. First, a critical literature review 
was conducted to compile a theoretical research 
framework consisting of indicators of system success 
and critical success factors that could be used as a basis 
to evaluate the ePortfolio implementations. In the 
second stage, the critical success factors in this 
framework were empirically verified in five case 
studies using qualitative research methods. 

Participants 

The five organizations studied include five private, 
commercial businesses in the southeast region of The 
Netherlands (also known as the Brainport region). 
These businesses voluntarily enrolled in the project in 
early 2012 and completed the project late in 2014. They 
were provided with an ePortfolio system of their 
choice, free of charge, to experiment with. Table 1 
provides a brief overview of their company size and 
number of participants in the pilot. The defined HCM 
goals and objectives and their outcomes for each of the 
five cases are presented in Appendix C. 

Materials and Procedure 

Stage 1. To establish whether the implementations 
of ePortfolio systems in the Let’s Connect project were 
successful, the evaluation criteria have to be decided 
on. The literature on executive information systems 
provides a set of criteria that has been thoroughly 
researched by various scholars. In a study by Poon and 
Wagner (2001), five main criteria were distilled from 
the literature: 

1. Access: the system is made available and users 
are given access to the system; 

2. Use: the system is used by the intended users; 

3. Satisfaction: users are satisfied with the 
system; 

4. Positive impact: the system has positive 
impact on the executives and the organization; 

5. Diffusion: the system tends to spread. 

To contextualize and explain these outcomes, a 
systematic analysis of the implementation process is 
required. First, Poon and Wagner’s (2001) five criteria 
are used to establish the degree of success in 
implementation and the suitability of ePortfolios for the 
workplace. Subsequently, to contextualize these 
findings and to explain success and failure, a 
framework of critical success factors is compiled. This 
framework consists of a combination of the EMM and 


ESF frameworks. Together, the two models share a 
degree of complementarity. The EMM model addresses 
factors that represent the unique characteristics of the 
ePortfolio by emphasizing such aspects as 
interoperability, autonomy in ePortfolio use, and the 
ownership of data. Furthermore, it consists of more 
generic factors related to the hosting institution and 
staff. These generic factors are covered more 
elaborately in the ESF framework, which has been 
validated through decades of empirical research. The 
two models’ frameworks were compared side by side, 
identifying unique and overlapping factors. This 
comparison resulted in the Combined Critical Success 
Factors (CSF) framework, as depicted in Table 2. The 
left and right columns include 19 unique factors from 
both sources, while nine overlapping factors are 
presented in the middle column. Factors from the EMM 
framework, which was originally designed for the 
educational context, have been reworded, where 
appropriate, to represent the workplace. 

Stage 2. To identify the critical success factors of 
ePortfolio implementations, five case studies were 
analyzed. After establishing whether an implementation 
was successful, the critical success factors determining 
success and failure were analyzed through primary and 
secondary data collection. To facilitate data 
triangulation, the questions were answered through an 
analysis of different sources, including project 
documentation (e.g., presentations, field notes, and 
meeting reports) and through semi-structured 
interviews conducted at the beginning and final stages 
of the project. The interviews were held with HR- 
personnel involved in the implementation process and 
with employees using the ePortfolio. To answer the first 
research question, the fulfilment of the five main 
success criteria was established by seeking answers to 
such questions as “Does the organization intend to 
continue with the ePortfolio after the pilot phase?” 
(representing diffusion) and “To what extent were pre¬ 
defined HCM goals and objectives reached?” 
(representing positive impact). To determine the critical 
success factors in all cases, all factors from the 
compiled framework are discussed as themes in a semi- 
structured interview. Examples of interview questions 
included “To what extent did you define system 
requirements for the ePortfolio?” and “In your view, 
does the ePortfolio belong to the organization or the 
employee?” In the search for patterns, the similarities 
and differences about relationships within the data are 
examined. Cross-case analysis is conducted to examine 
the identified CSFs. 

The method involves a content analysis to 
corroborate the compiled CSF framework and make 
adjustments where appropriate. Content analysis entails 
a systematic, rigorous examination of data that results 
in discerning themes (Marsh & White, 2006). As a 
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Table 1 


Participating Pilot Organizations 





Case studies 




Cl 

C2 

C3 

C4 

C5 

Sector of industry 

Materials 

Industrial 

Materials 

Information 

Technology 

100-250 

Materials 

Company size 
(# of employees) 

500+ 

500+ 

100-250 

0-10 

Participants 

70 

40 

58 

20 

7 


Table 2 


Critical Success Factors Framework for ePortfolio Implementations 


Critical success factor 

ESF 

Source 

EMM 

Overlap 

Committed and informed executive sponsor 

X 



Appropriate supporting staff 



Staff ICT Skills 


X 

X 

Staff engagement 




Staff providing feedback 

Operating sponsor 

X 



Appropriate technology 

x 

x 

Usability 




Reusability 

Management of data 



Connectivity 


X 

X 

Interoperability 




ICT policy 

Clear link to business objectives 

X 

X 

ePortfolio policy 

Management of organizational resistance 

X 



Management of system evolution and spread 

X 



Evolutionary development methodology 

X 



Carefully defined information and system requirements 

X 



Employee autonomy in learning 


X 


Employee autonomy in ePortfolio use 


X 


Electronic links to the organization 


X 


Access to ePortfolio 


X 


Employees as active users 


X 


Employees as seekers of feedback 


X 


Engagement of employees 


X 


Institutional embedding 


X 


ePortfolio ownership 


X 



Note. (Hartnell-Young et at, 2007; Poon & Wagner, 2001) 


result, a redefined CSF framework for ePortfolio 
implementations in organizations was developed. 

Results and Discussion 
ePortfolio Success 

Overall, the organizations experienced varying 
success rates in implementing the ePortfolio. 
Despite the fact that the ePortfolio was available 
without restrictions in most organizations in terms 
of accessibility, the system failed in all other areas 
in three out of five organizations (see Table 3). 


These areas are discussed in detail below for each 
case study. 

Case 1. The implementation was a failure in Case 
1 (Cl). In terms of accessibility, participants indicated 
that they had no problems accessing the system. The 
ePortfolio, over the course of a year, was used once by 
all participating employees. They completed their 
profile and did a standard assessment. Staff members 
who were responsible for on-the-job learning with the 
ePortfolio did not use or promote the system. Reported 
satisfaction with the ePortfolio was, however, high 
among all participating employees. They indicated that 
it could be a very useful tool for their professional 
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Table 3 

Pilot Outcomes 





Case Studies 



Cl 

C2 

C3 

C4 

C5 

Access 

± 

S 

✓ 

■/ 


Use 

X 

X 

± 

± 

X 

Satisfaction 

X 

X 

✓ 

■/ 

X 

Positive impact 

X 

X 

± 

■/ 

X 

Diffusion 

X 

X 

■/ 

■/ 

X 

Overall 

X 

X 

■/ 

■/ 

X 


Note. S = Successful, ± = Acceptable, x = Unsuccessful. 


development and careers and wished that the 
organization had made better arrangements to ensure a 
proper implementation. The ePortfolio ultimately did 
not have a positive impact; none of the HCM goals 
were reached, and only one of the six objectives was 
partially reached. Consequently, the ePortfolio use did 
not spread, and the system has been discontinued. 

Case 2. The implementation was a failure in Case 

2 (C2). In terms of accessibility, participants indicated 
that they had no problems accessing the system. The 
ePortfolio, over the course of 2 years, was used once by 
all participants, who completed their ePortfolio profile 
and a personality test. However, there was no follow-up 
to this by the organization, despite some employees 
showing interest in the system. Reported satisfaction 
levels with the ePortfolio were low. Some workers 
indicated that it could be a useful tool for their 
professional development and careers and thought of 
the ePortfolio as complementary to their Linkedln 
profile due to the personality test and assessments, if it 
were used actively in the organization. Others 
employees were skeptical of the ePortfolio and feared 
that the contents would be used in lay-off procedures. 
In addition, the HR manager was not satisfied with the 
quality of the assessments included in the system. The 
ePortfolio did ultimately not have a positive impact; 
none of the HCM goals or objectives were reached. 
Consequently, the ePortfolio use did not spread and the 
system has been discontinued. 

Case 3. The implementation was a success in Case 

3 (C3). In terms of accessibility, participants indicated 
that they had no problems accessing the system. All 
participants used the ePortfolio once. Reported 
satisfaction with the ePortfolio was high among all 
participating employees. They indicated that it was 
convenient to have the system integrated into the HR 
system they were already using. Consequently, they had 
a single-access point for all employment-related matters 
such as salary slips, completed training, and developed 
competences. The ePortfolio did have a positive impact; 
the two HCM goals were partially reached, as the 
process was on-going when this present paper was 


published. Both the HR manager and the participating 
employees believed that the introduction of the system 
contributed to talent management and mobility, and that 
effects will materialize in the long run. The 
organization has decided to use the ePortfolio as an 
inherent part of future performance appraisals; as such, 
the system has spread across the organization. Four of 
the eight objectives were reached; the remaining four 
are in progress. The organization decided to continue 
ePortfolio use in the organization after the project. 

Case 4. The implementation was a success in Case 

4 (C4). In terms of accessibility, all participants 
indicated that they had no problems accessing the 
system. The ePortfolio was used multiple times by all 
participants, who had feedback sessions with 
supervisors. Reported satisfaction with the ePortfolio 
was high among all participants. Both supervisors and 
employees indicated that the ePortfolio was extremely 
useful as a basis for performance appraisals. The HR 
manager was satisfied that they managed to structure 
the ePortfolio in accordance with existing competence 
profiles in the organization, which can enable the 
organization to make a quick scan of all present 
competences in the organization. The organization and 
its employees were positive that the ePortfolio will 
reach the HCM goals across the entire organization 
once its development is complete, which is illustrative 
of the positive impact of the ePortfolio. Furthermore, 
efforts are currently being made to collaborate with a 
local university to achieve consensus on a fixed set of 
competences that can then foster the recruitment of 
graduates through standardized competence profiles. 
Their ePortfolio system will then function as a linchpin 
between the organization and university. Three of six 
objectives were reached; the remaining three are in 
progress. The organization is continuing ePortfolio use 
after the project. 

Case 5. The implementation was a failure in Case 

5 (C5). In terms of accessibility, there were no 
experienced issues. The ePortfolio system was used 
once during a brief introduction by the HR manager. 
Users were not satisfied with the system and indicated 
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they were not interested in using it. The ePortfolio did 
not have a positive impact in the organization, none of 
the HCM goals was reached, and only one out of four 
objectives was accomplished (i.e., describing relevant 
competences for various functions). The organization 
decided to discontinue ePortfolio use. 

Implementation 

To contextualize these findings and to identify 
critical success factors that impacted success of 
implementation, all factors from the CSF 
framework were analyzed for each case study and 
compared across all cases (see Table 4). First, 
constant factors across all cases are discussed, 
followed by factors in which there were notable 
differences between successful (C3, C4) and 
unsuccessful (Cl, C2, C5) cases. 

Constant factors across successful and failed 
cases. In both successful and failed 
implementations, the moderate levels of employee 
autonomy in learning and ePortfolio use seemed to 
have been a constant. All organizations indicated 
that employees were provided with a certain degree 
of freedom in defining professional learning goals 
and using the ePortfolio, as long as these goals and 
activities were relevant to the function in which 
they were employed or to which they were aspiring 
to grow. The nature and contents of the assessments 
were dictated by the organization. As one HR 
manager stated, “We want to use the ePortfolio for 
performance appraisals; therefore, the ePortfolio 
contents and use must be related to competences 
which we find relevant for a particular function” 
(C3). This is necessary because the ability to 
benchmark qualitative ePortfolio data is inherent to 
nearly all formulated HCM goals and objectives, in 
line with the positivist perspective on ePortfolios 
(in which externally defined learning outcomes are 
crucial). Therefore, these factors can be considered 
irrelevant, as they are inherent to the workplace and 
human capital management. This factor originates 
from the ePortfolio Maturity Model, which was 
tailored to the educational setting and, as such, 
places more emphasis on the constructivist 
perspective (in which ePortfolio meaning is mostly 
limited to the individual). 

Furthermore, in all five cases the electronic links to 
the organization and access to the ePortfolio were 
mostly unhindered. The former is unsurprising, given 
the increasingly interconnected nature of the workplace 
through cloud-based applications; therefore, this factor 
can be considered outdated and irrelevant. As for access 
to the system, a few minor complications were reported 
with regards to login problems in the early phase of the 
pilot; however, these were resolved quickly, and 


participants were all able to access the ePortfolio 
whenever they wished to do so. 

A striking observation is that all five organizations 
suffered from the lack of a committed and informed 
executive sponsor. This is illustrated by the following quotes 
during the interviews with HR managers: “Executive 
management support merely consists of them allowing me 
to spend time on it” (Cl); “[The company executive] never 
has time to discuss this project with me; he initiated the 
project and delegated it to me” (C3); and, “The company 
CEO made arrangements to participate in this project and 
handed it over to [the HR department]” (C4). As a result, the 
responsibility for implementation rested with the supporting 
staff and operating sponsor in all organizations; however, in 
C5, due to the small size of the company, one person 
fulfilled all three roles. It seems that while the executives of 
the organizations were in favor of the project, their actual 
involvement was relatively hands-off. Arguably, this can be 
attributed to the subsidy-driven nature of the project, and the 
lesser urgency compared to intrinsic business needs. Three 
managers (Cl, C2, C5) similarly stated that they did not 
have time for it now, as the business had more urgent 
priorities. 

Factors fulfilled in successful cases and 
unfulfilled in failures. The presence of active 
information system (IS) support and an operating 
sponsor were two of 13 factors in which there are 
distinct differences between failed (Cl, C2, C5) and 
successful (C3, C4) cases. In the failed instances, both 
the HR manager and the employees reported that the 
ePortfolio was not actively managed by the 
organization. In Cl, employees indicated that the HR 
manager briefly introduced the ePortfolio to them, but 
that there was no follow-up, causing the participants to 
lose interest. In this case, the HR manager himself also 
admitted that the ePortfolio was low on his priority list 
due to the effects of the economic crisis on his 
organization and due to the fact that there was no 
operating sponsor who could manage the 
implementation. One important objective here was to 
strengthen the relationship with the in-house 
educational provider through the ePortfolio; in the 
interview, however, the manager stated that he gave up 
on this after the “educational institute didn’t call me 
back about it.” Confronted with the lack of progress the 
organization was making throughout the project, the 
manager insisted that it was “a complex project, which 
requires more time.” 

In C2, the HR manager indicated that she had lost 
interest due to an interplay of other factors: (a) 
promises about the ePortfolio’s functions were not lived 
up to, and the quality of standard assessments was 
disappointing (appropriate technology); (b) employees 
were skeptical about the organization’s intentions with 
the ePortfolio (managing organizational resistance); and 
(c) other staff members were struggling with the 



Lievens 


Human Capital Management 195 


Table 4 

Critical Success Factors Outcomes 


Case studies 

Critical success factor 

Cl 

C2 

C3 

C4 

C5 

Committed and informed executive sponsor 

X 

X 

X 

X 

X 

Appropriate supporting staff 

X 

X 

✓ 

✓ 

± 

Operating sponsor 

X 

X 

± 


X 

Appropriate technology 

s 

X 

✓ 


X 

Management of data 

X 

X 

✓ 

✓ 

X 

Clear link to business objectives 

X 

X 

✓ 

✓ 

X 

Management of organizational resistance 

X 

X 

✓ 

✓ 

X 

Management of system evolution and spread 

X 

X 

✓ 

✓ 

X 

Evolutionary development methodology 

X 

X 

✓ 


X 

Carefully defined information and system requirements 

X 

X 

✓ 

✓ 

X 

Employee autonomy in learning 

± 

± 

± 

± 

± 

Employee autonomy in ePortfolio use 

± 

± 

± 

± 

± 

Electronic links to the organization 

v' 



✓ 


Access to ePortfolio 

± 

s 

✓ 



Employees as active users 

X 

X 

✓ 


X 

Employees as seekers of feedback 

X 

X 

✓ 


X 

Engagement of employees 

X 

X 



X 

Institutional embedding 

X 

X 

✓ 


X 

ePortfolio ownership 

± 

X 

✓ 

✓ 



Note. S = Successful, ± = Acceptable, x = Unsuccessful. 


concept of competences (managing organizational 
resistance). In this case, an operating sponsor was also 
lacking. In C5, the company executive (also acting as 
IS support and operating sponsor), did not have 
sufficient time for the pilot: “In a small company such 
as mine, you either need someone who is fully 
dedicated to the implementation or a system which is 
instantly ready to use; I simply did not have the time.” 

In C3, the combined role of IS support staff and 
operating sponsor also rested with the HR manager; 
however, she did manage to accomplish a partially 
successful implementation. She indicated that although 
it was tough to handle the entire project, she 
systematically (albeit slowly) developed the system in 
accordance with the organization’s needs, gathering 
support by frequently discussing the project with 
employees and senior management. In an interview, she 
stated that the project would have developed more 
quickly if it had not rested completely on her shoulders. 
In C4, the role of IS support rested with the HR 
manager, and another HR staff member functioned as 
operating sponsor. They also held frequent meetings 
with employees and senior management to ensure that 
the ePortfolio was aligned with organizational needs. In 
both cases, this method of evolutionary development 
resulted in an absence of organizational resistance. 

The management of the system’s evolution and 
spread, in which the ePortfolio is tailored to the needs 
of the organization and its users, was lacking in the 


failed cases. In Cl and C2, employees indicated that no 
efforts were made by the organization to cater to their 
needs and that their feedback was not acted upon. In 
C2, an employee stated, “we thought the project was 
finalized 2 years ago; we did not hear anything from the 
organization since the system was introduced.” 
Similarly, in the case of Cl, employees were in 
agreement that the organization’s efforts were 
“lackluster.” In C5, the CEO also struggled to develop 
the ePortfolio, despite being aided by an additionally 
hired HR consultant halfway through the pilot. In the 
two successful cases, the operating sponsor held 
meetings with supervisors and employees to develop 
further the system. This resulted in C3 being successful 
by embedding the ePortfolio in the company’s existing 
HR software, and C4 switching to a different ePortfolio 
supplier after employees and staff complained about the 
quality of standard assessments of the previous system. 
For example, one staff member stated, “They were 
similar in quality to those you find in a magazine at the 
dentist’s office.” 

In the three failed cases, there was a lot of 
unmanaged resistance against the ePortfolio 
implementation. In Cl, employees were enthusiastic 
about the system’s possibilities; however, they 
expressed dissatisfaction with the way the system was 
introduced and distrust in the organization’s intentions. 
They indicated that an HR manager briefly introduced 
the ePortfolio to them but that there was no follow-up, 
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which led to the participants losing interest. 
Furthermore, participants seemed to be wary of the 
organization’s intentions. They expressed a fear of 
being monitored and thought the ePortfolio could be 
used against them in the case of lay-offs. It also became 
apparent that employees had to request permission from 
the HR manager to share their ePortfolio contents with 
others. This reinforced their suspicions about being 
monitored and their reserved attitude. Furthermore, 
staff members responsible for on-the-job training were 
known to resist the ePortfolio system, and the HR 
manager did not address the issue. 

In C2, there also was a group of employees who 
feared that ePortfolio contents would be used in lay-off 
procedures. In this respect, the organization admitted 
that the introduction of the ePortfolio system and its 
purpose to this group was flawed. This occurred in the 
beginning of the project and was reported to have 
impacted the reputation of the project through word of 
mouth. Furthermore, the HR manager was dissatisfied 
with the purchased ePortfolio system (a decision made 
by the corporate executive), which did not meet her 
expectations. She felt that the quality of the assessments 
included was sub-par and that a lot of work and time 
were required to adapt the ePortfolio to the 
organization. In C5, this resistance was caused by 
employees uninterested in the system, who stated that 
they did not understand what it was about and why it 
was needed. The company executive admitted that he 
had failed to gather support for the system. He 
attributed this to a lack of time, which resulted in a slow 
customization of the system, tailored to the company’s 
needs (a lack of evolutionary development). In addition, 
he said that in hindsight, he wished he had considered 
other ePortfolio systems to compare customization 
options and user-friendliness, which he felt were 
lacking in his system. 

The choice of an appropriate ePortfolio system 
turned out to be of crucial importance in three other 
cases. In C2, the ePortfolio was considered unsuitable 
to meet the company’s HCM goals and objectives. The 
competence assessments lacked substance and were 
“more suitable for orientation purposes rather than 
evidence-based decision making.” The HR manager 
said, “I feel like this ePortfolio system prioritized the 
technology over the quality of the competence 
assessments; it is not suitable for our needs.” The HR 
manager in C4 had a similar stance but took efforts to 
switch to a more appropriate system. In these cases, the 
factor “management of data” was of great importance; 
the systems lacked the ability to provide access to 
reliable data on the employees’ competences. 
Furthermore, the ePortfolios lacked sophisticated 
importing and exporting functionalities, which resulted 
in organizations being unable to exchange competence 
data with educational institutes. This can also be 


explained by the highly contextual nature of 
competences, which hindered standardization. The 
existing IMS standard and its derivatives do not 
facilitate the exchange of competence data. There are 
developments in this field, such as 0*NET in the 
United States (including descriptions of competences 
related to various occupations) and the European 
Qualifications Framework; however, these have not yet 
been applied to an ePortfolio infrastructure. C4 is 
continuing to pursue this after the project by using an 
ePortfolio system that connects the company with a 
local university. This idea is similar to an Italian 
ePortfolio platform connecting the workplace and 
education, AlmaLaurea, which consists of a database of 
student ePortfolios from which companies can recruit 
graduates. C4 is making efforts also to include 
standardized competences to the platform they 
envisage. 

In C3, no separate ePortfolio system was 
purchased; instead, the existing HR tool was 
customized to include desired ePortfolio functionalities 
(competence profiles of each employee). Only in Cl, 
the system’s choice did not have a significant impact. 

In the two successful cases, there was a clear link 
between the ePortfolio and the business objectives. In 
C3, the HR manager stated, “Now that the ePortfolio’s 
development is complete and that all our company’s 
competence profiles are included, we can use the 
profiles of our employees as input for performance 
appraisals.” In C4, the ePortfolio’s added value also 
stemmed from the system allowing the organization to 
benchmark employees (in terms of their competences) 
and to start using the system for recruitment purposes in 
cooperation with a local university. The management of 
data was inherent to the successful linkage of the 
ePortfolio with business objectives. In turn, the 
successful management of data was dependent on a 
careful definition of information and system 
requirements. While in a broad sense all five 
organizations defined HCM goals and objectives, only 
in the two successful cases were efforts made to 
identify the exact information required from the 
ePortfolio to fulfill these. In C3 and C4, relevant 
competences for each participating function in the pilot 
were identified and described. In addition, meetings 
were held to identify the needs of all staff members that 
would be using the system. This information was 
crucial for customizing the ePortfolio in a manner that 
allowed it to be implemented effectively. 

In two failed cases, ePortfolio ownership had a 
detrimental effect on the evolution and spread of the 
system. In Cl, employees reported that they had to ask 
permission from their supervisor to share ePortfolio 
contents with others. Therefore, they felt that the 
system was only being used to monitor the employees 
and that they had limited freedom in the way it could be 
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used. The HR manager was unaware of this, indicating 
that this was an incorrect interpretation by the 
employees; however, the manager did not undertake 
actions to take away these concerns. In C2, the 
company executives expressed worry that the ePortfolio 
could be harmful for the organization, as it could 
facilitate the headhunting of talented employees by 
other organizations. Therefore, they insisted that 
ePortfolio ownership rested with the organization and 
that employees would be limited in sharing their 
profile. 

The institutional embedding of, engagement with, and 
use of the ePortfolio by employees (e.g., activity and 
seeking feedback) was negative in the failed cases and 
positive in the successful ones (except for C5, in which case 
the company executive struggled to explain what the system 
was and why it was being introduced). Employees were 
unanimously positive about the concept of ePortfolio, but 
dependent on the implementation by senior management 
because this was a top-down organizational process. In each 
of the five cases, a combination of the factors discussed 
above resulted in a lack of use and engagement by the 
employees and in a lack of institutional embedding. As 
such, they were not of critical importance in these cases and 
can be considered as outcome measures rather than 
conditions for the implementation. 

Judging by the outcomes, ePortfolio implementations 
in organizations are likely to be a best practice rather than a 
best fit. The processes show significant similarities in terms 
of factors that made a difference between success and 
failure. In each successful case, a different interplay of a 
constant set of factors made a difference. The successful 
cases demonstrated that linking the ePortfolio to business 
objectives by using appropriate technology, carefully 
defined information requirements, and an evolutionary 
development methodology with committed and informed 
staff led to a successful implementation. The failed cases 
suffered from a lack of informed and committed staff, which 
in turn resulted in a lack of information requirements, poor 
management of organizational resistance, and inappropriate 
technology being used. Furthermore, the cases in which 
ePortfolio ownership mostly remained with the employees 
led to an uptake in use and engagement. C5 was the least 
successful case, which can be explained by the relatively 
small company size compared to the other cases and the 
lack of HR expertise to implement a new information 
system. These observations result in the revised framework 
of critical success factors for ePortfolio implementations in 
organizations, as depicted in Table 5. 

Limitations 

One limitation of this study is related to 
generalizability. The sample size was relatively small. 
Furthermore, there were large differences between the 
five organizations. They all varied in company size, 


operated in different sectors, and faced different 
external pressures (e.g., the recession) impacting the 
time and resources allocated to this project. These 
contextual factors may have accounted for the different 
outcomes. However, despite these differences the CSFs 
identified shared a large degree of similarity across all 
cases and as such we have a deepened understanding of 
ePortfolio applications in the workplace. 

Another limitation of this study is the external 
financial incentive for participation. All costs related to 
the purchase and use of ePortfolio systems were 
covered by the project. It is a possibility that 
implementation outcomes, as well as the 
accomplishment of critical success factors were 
affected by this. 

Conclusion 

This study aimed to investigate whether the 
ePortfolio is a suitable instrument for the workplace. 
This was researched by an analysis of a three-year case 
study that took place in The Netherlands. While the 
ePortfolio did not fully realize its potential in terms of 
facilitating mobility and life-long-learning, the two 
successful cases show that the ePortfolio can be a 
valuable instrument in the process of internal human 
capital management. In both cases, the system 
facilitated the gathering of qualitative data on 
competence mastery of employees that can be used for 
performance appraisals and to identify talented workers 
by comparing their competence profiles, in line with the 
theoretical assumptions on the utility of the ePortfolio. 
Organizations failed to exchange ePortfolio data with 
educational institutes to foster recruitment. This can be 
explained by the fact that the ePortfolios were used in 
isolation in the organizations, and that the import and 
export functionalities suffer from a lack of 
standardization. 

Three out of the five cases failed to implement the 
ePortfolio, which is illustrative of the complexity that 
surrounds the implementation of such an information 
system. The only constant positive success indicator 
across all five cases was providing access to the system. 
To explain these findings, an analysis of the 
implementation processes in each of the five cases by 
means of a compiled theoretical framework of CSFs 
followed. This resulted in the identification of 11 
critical success factors that impacted the ePortfolio 
implementations in this case study (Table 5). These 
factors can be summarized as (a) linking the ePortfolio 
with business objectives, (b) carefully identifying 
information requirements and selecting a suitable 
system, (c) actively managing the implementation by 
appropriate and dedicated staff throughout the 
organization, and (d) ensuring the employees have 
ownership over their ePortfolio profiles. In all five 
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Table 5 

Critical Success Factors Framework for ePortfolio Implementations in Organizations 

Critical success factor 


Committed and informed executive sponsor 
Appropriate supporting staff 
Operating sponsor 
Appropriate technology 
Management of data 

Clear link to business objectives_ 


Management of organizational resistance 
Management of system evolution and spread 
Evolutionary development methodology 
Carefully defined information and system requirements 
ePortfolio ownership 


cases, a combination of these factors determined failure 
or success. 

This study contributed to the literature on 
ePortfolio use by investigating empirically theoretical 
claims about the utility of the concept in the workplace. 
This further advances the knowledge on the different 
applications of the concept. Furthermore, it provides 
organizations with a framework of critical success 
factor that can be used to plan an effective ePortfolio 
implementation. The Let’s Connect program ultimately 
aimed to enhance mobility of workers. The research 
reveals that the ePortfolio’s suitability for usage across 
different contexts is limited, due to the inability to 
exchange qualitative data in a uniform manner. This 
suggests that the ePortfolio can only perform 
effectively in a platform-function in which data is 
interpretable and exchangeable by all parties. Such a 
platform could not be realized within the timeframe of 
the Let’s Connect program. Future research could 
investigate the effects of ePortfolio use in such a 
platform-function (e.g., AlmaLaurea), in which the 
exchange of qualitative data across different contexts 
takes place. Individuals and organizations that are part 
of such a platform could be followed to establish 
whether the ePortfolio fosters mobility. 
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Appendix A 

The ePortfolio Maturity Model 


_ Maturity factor _ 

ePortfolio Policy 

Connectivity to support ePortfolio 
development 

Interoperability/transferability of data 

Curriculum ICT Policy 
Institutional embedding 


_ Description _ 

The institute has an articulated policy relating to ePortfolio purpose, 
use, and development. 

Systems are networked together to allow the sharing of ePortfolio 
resources. 

The system offers flexibility with regards to the import and export of 
data. 

A clear vision has been defined on the use of ICT in the institute 
Acceptance of the ePortfolio in the institute. 


Staff ICT Skills 

Staff engagement to ePortfolios 
Staff as providers of online feedback 

Autonomy in the construction of 
ePortfolios 

Student capability of autonomy in 
learning 

Student’s electronic links to the 

organization 

Access to ePortfolio 

ePortfolio ownership 

Learners as active creators of digital 

content 

Learners as seekers and users of feedback 
Learner engagement to ePortfolios 
Usability 

Reusability _ 

Note. (Hartnell-Young et at., 2007) 


The majority of staff is ICT affluent. 

Engagement is universally positive. 

Staff work regularly, constructively, and formatively on giving 
feedback on ePortfolio material. 

Students autonomy is encouraged as a matter of policy. 

All students are capable of making autonomous choices regarding their 
learning goals and style. 

Students can access the ePortfolio from home. 

The ePortfolio is available anywhere, anytime. 

Students can decide which aspects of the ePortfolio are shared. 

Students are active and regular creators of content. 

Students seek feedback regularly. 

Engagement is almost universally positive. 

The interface is well designed and intuitive. 

Any agreed type of data/file can be stored. _ 
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Appendix B 

Executive Information System Success Factors 


_ Critical success factor _ 

Committed and informed executive sponsor 

Operating sponsor 
Appropriate supporting staff 
Appropriate technology 

Management of data 

Clear link to business objectives 

Management of organizational resistance 


Management of system evolution and spread 

Evolutionary development methodology 
Carefully defined information and system 
requirements _ 

Note. (Poon & Wagner, 2001) 


_ Description _ 

Executive sponsor who is committed to the implementation, 
invests time and effort and has a realistic understanding of the 
system. 

Operating sponsor who actively manages the implementation 
and its details, to leverage the time of the executive sponsor. 
Supporting staff who have technical as well as business 
knowledge to support the implementation of the system. 

Selecting the most suitable system on the market, which is 
crucial since the choice has a major bearing on the acceptance of 
the system. 

Ability to provide access to reliable data from internal and 
external sources. This may involve the aggregating, accessing 
and extracting data from various databases. 

The benefits of the system and link to a certain business 
problem / objective are clearly defined. The system should 
provide something that adds value. 

Proactively managing organizational resistance in the 
introduction and operational phase, which is a common cause of 
implementation failure. 

Identifying specific job functions, technical orientation, work 
style and support needs of each user. 

Prototyping to discover how the system can add value. 
Identifying information requirements that meet the 
organization’s needs in terms of the defined objectives. _ 
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Appendix C 

Goals and HCM Objectives for the Five Cases 


Cl 


Result 


HCM goals 


Objectives 


Shift to recruiting personnel based on competences (clusters of knowledge/skills/attitudes 
which enable a worker to work effectively). 

Structuring training and on-the-job learning with the ePortfolio by using elaborate 
competences. 

x Recording and monitoring process of in-house education with the ePortfolio. _ 

± Describing 16 job functions in terms of primary tasks in alignment with educational records 
Completed ePortfolio profiles of all employees currently involved in (internal or external) 
education. 

Identifying competences relevant for the different job functions and embedding them in the 
ePortfolio. 

x Gaining insight in the talents of employees. 

x Shortening duration of training due to enhanced insight in competence mastery levels. 
Expanding the number of participants to 200 in 2 years’ time. 


NoteS = Successful, ± = Acceptable, * = Unsuccessful. 

C2 

Result 


HCM goals 

X 

Improve the composition of teams based on competence assessments in the ePortfolio 


X 

Fostering internal mobility (redeploying personnel in positions appropriate to their 
competences) through identifying talented employees based on ePortfolios 

Objectives 

X 

Develop a valid competence test to be used in the ePortfolio 


X 

Completed ePortfolio profiles of all participants 


X 

Gain insight in the competences of participants on an aggregate level 


X 

Expand the number of participants 

Note. S = Successful, ± 

= Acceptable, x = Unsuccessful. 

C3 

Result 


HCM goals 

± 

Gain insight in professional development of employees. 


± 

Fostering (internal and external) mobility of employees. 


Objectives 


✓ 

✓ 

± 

✓ 

± 

± 

± 


A description of all job functions, roles, tasks and associated competences. 

Completed ePortfolio profiles of all participants. 

Including ePortfolio training in the company training. 

Embedding the ePortfolio in the existing HR software. 

Active use of ePortfolios among employees. 

Gain insight in the talents of employees. 

Expand the number of participants to 120. 

Exchanging data on requested competences with the vocational institute where employees 
are recruited from. 


Note, v' = Successful, ± = Acceptable, x = Unsuccessful. 
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C4 _ Result _ 

HCM goals ± Fostering (internal and external) mobility of employees. 

± Improve the composition of teams based on competence assessments in the ePortfolio. 

_±_ Stimulating professional development of employees. _ 

Objectives ^ Completed ePortfolio profiles of all participants. 

'f Active use of the ePortfolio system by both the participants and supervisors 
± Insight in the competences of participants on an aggregate level. 

± Developing a competence test which is linked to relevant job functions in the pilot. 


Exchanging data on requested competences with a local university to recruit graduates more 
effectively. 

Embedding existing competence profiles of the organization in the ePortfolio. _ 


Note. ^Successful, ±=Acceptable, *=Unsuccessful 

C5 

Result 


HCM goals 

X 

Enhancing the professional development of employees, 


X 

Stimulating mobility of employees. 

Objectives 

X 

Describing relevant competences for the job functions in alignment with vocational training 
institutes. 


X 

Recording both the vocational generic competence levels of employees in the ePortfolio. 


Note. S = Successful, ± = Acceptable, * = Unsuccessful. 



